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Abstract 


Oats  Envalopaant  Analysis  is  a  relatively  new  Method  for  Measuring  and 
evaluating  the  efficiency  of  not-for-profit  entities  with  Multiple  outputs 
and  Multiple  inputs.  Without  requiring  a  priori  weights  and  without 
requiring  explicit  specif ication  of  interdependencies  that  aey  be  present 
between  variables  it  (a)  provides  an  overall  Measure  of  performance 
efficiency  fron  observational  data  and  (b)  identifies  sources  and  estimates 
asounts  of  inefficiency  that  soy  be  present  in  each  such  source.  This  paper 
reports  on  results  froa  studies  of  DEA  for  its  possible  use  in  evaluating 


tha  perforsance  of  24  Army  Health  Care  Facilities.  These  OEA  results  are 
reviewed  and  compared  with  results  fro®  alternate  Methods  (often  used  for 
efficiency  analysis)  such  as  statistical  regressions  and  standards  imposed 
by  HCFA . 


&8.Y  VteEtit. 

Health  Care  Facilities 
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1.  BACKGROUND 

There  ia  a  continuing  interest  in  the  productivity  of  US  Army  health 
care  facilities  which  has  intensified  in  the  past  two  years.  New  and  im¬ 
proved  ways  of  evaluating  performances  are  therefore  being  considered. 
Prior  to  1957,  average  daily  bed  occupancy  was  usad  to  evaluate  the  size  and 
output  of  Army  hospitals,  but  important  problems  existed  with  this  measure. 
First,  beds  occupied  measures  only  the  average  nusber  of  beds  utilized. 
This  measure  did  not  account  for  the  turnover  created  by  patients  being 
admitted  and  discharged,  even  though  this  turnover  represented  a  significant 
consumption  of  resources.  A  second  and  even  more  serious  problem  related  to 
the  vast  amount  of  ambulatory  care  being  performed  in  military  health  care 
facilities  and  this,  too,  not  explicitly  accounted  for  when  using  beds 
occupied. 

In  1957,  a  single  output  measure  was  developed  called  the  "Care  Com¬ 
posite  Unit".  This  CCU  Unit  was  developed  to  account  for  the  ambulatory 
care  being  performed  ana  the  turnover  of  inpatients.1  The  composite  unit 
used  admissions  as  a  sux'rr.g&ta  for  turnover.  In  this  way  one  of  the  inade¬ 
quacies  in  the  use  of  average  daily  beds  could  be  remedied.  But  it  was  also 
necessary  to  reflect  the  amount  of  ambulatory  care  explicitly  and  for  this  a 
weighted  meaaure  for  t!.a  ambulatory  care  was  used.  When  finally  developed, 
the  Care  Composite  Unit  weighted  average  daily  beds  occupied  by  one  and 
added  tan  times  the  number  of  average  dally  admissions  and  0.3  times  the 
average  daily  clinic  visits.  Originally,  these  weights  related  well  across 
the  entire  system  at  the  rate  of  100  personnel  per  100  Care  Composite  Unite. 
During  three  major  validation  attempts  in  the  past  26  years,  CCU  showed  high 

1In  civilian  hospitals,  ambulatory  care  was  handled  by  physicians  in 
their  offices. 
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correlations  to  the  total  system  manpower.  Over  the  past  tan  years,  how¬ 
ever,  a  major  shift  has  occurred  fro*  hospitalization  to  treatment  on  an 
ambulatory  basis  and  this  has  led  to  a  reconsideration  of  the  CCU  index. 

As  technology  has  advanced,  imbalances  became  acre  prominent  and  it 
baca*e  increasingly  apparent  that  all  clinic  visits  should  not  be  credited 
with  the  same  weight  nor  should  all  inpatients  be  credited  at  the  same  value 
during  their  stay.  In  searching  for  alternatives  and  improvements  it  was 
possible  to  turn  to  results  fro*  the  Health  Care  Financing  Agency  (HCFa) 
which  had  decided  to  reimburao  on  the  basis  of  weighted  Diagnostic  Related 
Groupings  (DRGs)  for  inpatient  care  provided  to  Medicare  patients.  Although 
the  DRG  weights  were  specifically  established  for  Medicare  patients,  they 
did  provide  a  standard  for  a  starting  point  in  identifying  differences  in 
the  consumption  of  resources  based  upon  patient  diagnoses.  Under  DRG  a 
discharged  patient  is  classified  into  a  group  on  the  basis  of  a  primary 
diagnosis  and  the  procedures  utilized  during  the  stay.  When  all  such 
weighted  dispositions  are  summed  for  a  period  Lne  result  is  called  the 
"Relative  Weighted  Product"  (RWP)  and  serves  as  a  relative  measure  of  the 
resources  consumed  by  inpatients. 

This  still  left  the  special  features  of  the  Army's  ambulatory  care 
activities  to  be  attended  to.  Currently,  the  Army  is  developing  a  syc.ee* 
for  ambulatory  patieuts  based  upon  a  patient  encounter  reporting  system 
being  tested  at  six  Army  facilities.  Results  of  this  study  should  be 
available  by  late  1386. 

The  traditional  approach  to  studying  the  properties  of  such  measures 
and  reporting  systems  has  bemn  via  regression  analysis  or  regression-related 
correlation  analyses.  Difficulties  arise  very  quickly  in  the  use  of  such 
analytical  instruments  because  of  the  complex  nature  of  modern  health  care 
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delivery.  Standard  regressions  of  t.he  single  aquation  at  a  time  variety 
(with  a  single  dependent  variable  in  each  equation)  say  miss  or  mack  inter¬ 
relations  between  outputs  and  between  inputs  as  well  as  between  inputs  and 
outputs.  These  relations  are  often  oc.wpJa:'  and  difficult  to  aodel  in  the 
forms  required  by  standard  regression  equation  formate.  For  instance, 
inpatient  care  is  often  directly  related  to  outpatient  cere  and  resources 
used  in  one  sraa  may  contribute  to  outputs  in  another  area  in  many  different 
(often  unknown)  ways. 

Data  Envelopment  Analysis  (PEA),  a  recently  developed  method  of  ef¬ 
ficiency  evaluation  and  analysis  developed  by  Charnes,  Cooper  and  Rhodes 

i 

[123  and  [13]',  offered  a  possible  way  of  dealing  with  theeo  difficulties. 
Designed  specifically  for  uae  in  not-for-profit  entities  with  multiple 
outputs  and  multiple  inputs,  DEA  does  not  x-equire  explicit  identif ication 
and  formulation  of  the  x'elations  that  might  obtain  between  the*.  Unlike 
index  numbers  and  other  related  attempts  to  obtain  relative  measures  of 
efficiency,  DEA  does  not  require  a  priori  weights  or  other  such  devices  to 
arrive  at  overall  efficiency  evaluations.  It  alio  estimates  amounts  of 
inefficiency  and  identifies  their  possible  sources. 

Although  the  product  of  relatively  recent  research,  DEA  has  now  been 
tested  in  a  variety  of  different  contexts  --  auch  as  those  reported  in  C51 
and  C9]  --  and  it  therefore  s named  worthwhile  to  undertake  further  study  and 
testa  of  DEA  for  poeaible  uae  in  dealing  with  problems  like  those  we  have 
already  mentioned  in  evaluating  U.S.  Army  heeltn  cere  facility  activities. 


"The  numbers  in  square  brackets  are  keyed  to  the  references  that  appear 
at  tha  end  of  this  papar. 
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2.  REPORT  FORMAT 


Table  1  will  help  to  sake  our  subsequent  discussion  more  concrete  and 
meaningful.  It  represents  a  report  generated  In  the  course  of  a  study  of 
possible  uses  of  DEA  undertaken  jointly  by  the  Aray  Health  Services  Coaaend 
at  Fort  Saa  Houston  in  San  Antonio  with  the  Center  for  Cybernetic  Studies  at 
The  University  of  Texas  at  Austin. 

This  study  was  based  on  reports  from  24  Aray  hospitals  and  other  fa¬ 
cilities  in  each  of  eight  quarters,  froa  first  quarter  1983  through  fourth 
quarter  1384.  In  OEA  terminology  each  such  facility  is  referred  to  as  a  DMU 
(Decision  Making  Unit)  indicating  that  it  is  responsible,  at  least  in  part, 
for  the  inputs  utilized  and  outputs  produced.  In  this  "prototype  study" 
three  outputs  and  eight  inputs  were  utilized  and  their  values  as  reported 
for  this  particular  DKU  <-  DMU  24)  are  shown  under  the  coluan  labelled 
"VALUE  MEASURED"  in  Table  1  aa  follows; 

a.  OUTPUTS 

(1)  PERS  TRAIN  -  Personnel  Trsined.  This  included  sll  personnel 
trained  during  the  quarter  excep',  for  residents  and  interns. 

(2)  RWP  *  Relative  Weighted  Product.  This  is  ths  RWP  defined 
above  for  which  Inpatient  data  used  the  dispositions  reported 
in  ths  Department  of  Defense  u'nifora  Chart  of  Accounts  Systea 
at  the  departaent  lavel  (two  digit  code)  aultiplied  by  the 
HCFA  weighted  departmental  DRG  Cass  Mix  Index  computed  from 
the  Aray  Inpatient  Data  System  (IPDS)  clinical  abstract 
record. 

(3)  CL  VISITS  *  Clinic  Visits.  These  data  represent  the  total 
number  of  encounters  made  by  patients  with  an  ambulatory 
clinic. 

b.  INPUTS 

(1)  CAT  1  FTE  *  Physicians  and  Other  Primary  Care  Providers  (cat  1 
and  2) .  These  data  were  derived  froa  the  Departaent  of 
Defense  Uniform  Staffing  Methodology  (U5M)  data  base  and 
equal  the  average  daily  number  of  full  t.iae  equivalents  (FTE) 
of  physicians  and  "physician  extenders"  for  tiae  spent 
treating  patients. 
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(2)  CAT  3  FTE  =  Registered  Nurses.  The  average  daily  nusber  of 
full  time  equivalent  registered  nurses  utilized  to  treat 
patients  during  the  quarter. 

(3)  CAT  4  FTE  =  Nursing  and  Adainiatrative/Logiatical  Personnel 
(cat  4  and  5) .  The  average  daily  number  of  full  time  equiv¬ 
alents  (other  than  categories  1  through  3)  utilized  in  direct 
patient  care  during  the  period.  These  data  do  not  include 
support  or  overhead  personnel  such  as  laboratory,  pharmacy, 
radiology  or  top  managment  personnel. 

(4)  INPAT  DLLS  =  Inpatient  Expenditures.  The  average  daily 
dollars  spent  by  the  facility  to  treat  all  inpatients, 

(5)  OUTPAT  DLLS  =  Outpatient  Expenditures.  The  average  daily 
dollars  spent  to  treat  all  asbulatory  patients. 

(6)  WT  PROCED  =  Weighted  Procedures.  The  average  daily  nusber  of 

weighted  procedures  accomplished  in  the  ancillary  work  centers 
utilizing  the  following  weights  from  the  normalized  weights  of 
the  American  College  of  Pathologists:  Radiology  (1.4977) 

Pharmacy  (.5717)  and  Pathology  (1.00). 

(7)  BED  DAYS.  The  average  number  of  daily  beds  occupied. 

(3)  OR  Hui/ES  =  Operating  Hours.  The  average  daily  number  of  hours 
the  operating  rooms  were  used. 


The  actually  reported  results  for  each  of  the  above  categories  for  DNU 
24  appear  in  the  column  under  “VALUE  MEASURED".  We  may  then  turn  to  the 
column  in  Table  1  labelled  "VALUE  IF  EFFICIENT".  Considering  the  behavior 
of  all  DMUa  in  a  manner  that  wi  chill  shortly  indicate,  DEA  identifies  only 
one  inefficiency  among  tha  out pit a.  The  estimated  amount  of  this  inef¬ 
ficiency  la  shown  undar  the  column  headed  "SLACK"  at  o  value  of  6.9  units  in 

the  row  for  PERS  TRAIN.1  That  is,  instead  of  training  only  36.0  personnel 
this  DMU  should  have  bean  able  to  train  42.9  («36  +  6.9)  parsons  if  it  were 
operating  efficiently.  See  the  42.9  opposite  PERS  TRAIN  under  tha  column 
headed  "VALUE  IF  EFFICIENT". 


Actually  the  data  in  this  table  have  been  multiplied  by  a  masking 
factor,  as  is  allowad  by  DEA  becausa  the  efficiency  ratings  are  invariant  to 
the  units  of  measure  used.  See  CO] . 
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Turning  to  tha  inputs.  DEA  identifies  and  estimates  several  further 
inefficiencies.  For  instance,  with  efficient  operations  (as  identified  by 
DEA)  DMU  No.  24  should  have  been  able  to  reduce  its  CAT  1  FTE  from  317  to 
797.5.  This  estimate  is  derived  from  the  overall  efficiency  value  of  0.976 
noted  at  the  top  of  the  table  by  applying  it  to  the  observed  value  of  817 

FTE  to  obtain  0.976  x  817  =  797,5.^ 

The  overall  efficiency  rating  of  0.976  applies  to  all  of  the  inputs  in 
the  same  manner  as  was  just  noted  for  CAT  1  FTE.  In  some  cases,  however, 
still  further  input  inefficiencies  are  identified  by  DEA  and  listed  under 
the  column  headed  ••SLACK".  Thus  to  obtain  tha  607.2  "VALUE  IF  EFFICIENT" 
for  CAT  3  FTE,  it  is  not  sufficient  simply  to  multiply  the  717  in  the  column 
"VALUE  MEASURED"  by  0.976.  It  is  also  necessary  to  reduce  the  resulting 
value  by  the  92^3  SLACK  that  is  present  in  this  input  to  achieve 

(0.S76  x  717)  -  92.8  *  607.2 

as  shown  for  CAT  3  FTE  in  the  column  labelled  "VALUE  If  EFFICIENT". 


^The  symbol  is  used  to  mean  "approximately  equal"  to  allow  for  computer 
roundoff  and  like  aberrations. 


TABLE  1, 


EFFICIENCY  = 
FACET: 

LAMBDA  = 


*  OUTPUTS  * 

PERS  TRAIN 
RWP 

CL. VISITS 


*  INPUT  * 


CAT  1  FTE 
CAT  3  FTE 
CAT  4  FTE 
INPAT. DLLS 
OUTPAT.DLLS 
WT  PROCED. 
BED  DAYS 
OR  HOURS 


A  DEA  Report 


DMU  NO.  24 


0.376 


71 

0.378 


37  95  28 

0.224  0.254  0.313 


45  20 

0.132  0.148 


VALUE  VALUE  IF 

SEASyRP _ EFFICIENT 


SLACK 


36.0 

42.9 

6.9 

238.0 

238.0 

o 

12733.0 

12733.0 

0 

817.0 

797.5 

o 

717.0 

607.2 

92.8 

2207.0 

365440.0 

2154.4 

IQ  ^ 

0 

394297.0 

<#ww-«4  y  ■  4. 

384905.4 

63x6.5 

0 

215020.0 

139079.4 

70819.2 

1397.0 

1363.7 

0 

436.0 

425.6 

0 
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How  does  DEA  arrive  at  these  resuits?  Exact  mathematical  character¬ 


izations  and  related  proofs  and  interpretations  nay  be  found  in  CSi  cr  CIO] , 
Hare  we  only  try  to  provide  a  “feel"  for  the  process  in  the  following 
manner. 

As  already  noted,  the  present  study  involved  24  Army  Health  Care  fa¬ 
cilities  for  which  complete  data  were  available  fro*  the  first  quarter  of 
1983  through  the  fourth  quarter  of  1984.  The  data  under  "VALUE  MEASURED"  in 
table  1  are  reported  activities  for  DMU  24  in  some  particular  quarter  and 
similar  data  were  available  for  each  of  the  other  DMUs  included  in  the 
study. 

The  mathematical  processes  and  computer  codes  used  by  DEA  search 
through  all  DMUs  in  order  to  pick  a  "best"  subset  for  evaluating  each  of 
them.  The  subset  thus  picked  by  DEA  for  evaluating  DMU  24  in  Table  1  is 
printed  at  the  top  of  Table  1  opposite  the  word  FACET.  The  components  in 
this  subset  are  identified  as  DMUs  71,  87,  95,  28,  45,  and  20  --  with  these 
numbers  resulting  from  the  way  the  individual  DMUs  were  coded  for  use  in  the 
"window  analysis"  in  section  4,  below. 

These  Decision  Making  Units  are  all  efficient,  as  determined  by  DEA, 
and  hence  generate  an  efficiency  "facet"  for  effecting  the  efficiency 
evaluation  exhibited  in  Table  1.  Other  efficiency  facets  can  also  be 
generated  by  using  other  sets  of  efficient  Decision  Making  Units,  but  this 
particular  facet  is  chosen  by  DEA  for  evaluating  DMU  24  because  it  is  the 
one  that  gives  this  DMU  the  highest  efficiency  score  that  the  data  will 
allow.  In  fact  the  point  consisting  of  the  output  and  input  values  that 
result  from  applying  the  "laebda"  values  0.378,  0.224,  0.214,  0.313,  0.132, 
and  0.148  to  the  reported  input  and  output  values  for  the  DMUs  under  which 
they  are  listed  in  Table  1  provides  the  point  of  comparison.  That  is,  these 
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lambda  valuta,  aa  determined  by  DEA,  are  used  to  locate  a  comparison  point 
in  this  facet  that  is  used  to  obtain  a  relative  efficiency  rating  for  DMU 
24.  Mo  other  point  in  thia  facet,  and  a  fortiori  no  point  in  any  other 
efficiency  facet  can  give  a  higher  efficiency  value  to  DMU  24  than  the  0.376 
value  listed  at  the  top  of  Table  1.  In  other  words  thia  is  the  highest 
rating  that  the  data  allow  for  scoring  the  efficiency  of  DMU  24  in  this 
quarter. 

The  efficiency  rating  for  DMU  24  in  Table  1  is  obtained  by  comparing 
its  behavior  with  the  behavior  of  other  DMUs  that  are  available  for  thia 
purpose.  The  value  0.976  thus  represents  a  "relative"  efficiency  rating. 
Only  Decision  Making  Units  which  have  relative  efficiency  ratings  of  1003 
can  be  used  to  fori*  these  facets  and  the  comparison  points  from  which  the 
DEA  evaluations  are  aadc.  It  is  against  this  same  point  on  the  efficiency 
surface  that  the  alack  values  are  also  determined  and,  as  we  shall  later 
see,  other  information  such  aa  “returns  to  3cale"  possibilities  may  al30  be 
obtained  from  the  lambda  values  in  Table  1. 

All  of  the  above  (and  more)  is  accomplished  by  DEA  and  the  computer 

codes  used  for  it3  implementation.^  Here  we  need  only  note  that  the  ef¬ 
ficiency  rating  aa  well  aa  the  sources  and  amounts  of  inefficiency  estimated 
by  DEA  may  be  validated  not  only  by  reference  to  the  individual  DMU  being 
evaluated  but  also  by  reference  to  the  DMUs  with  which  it  is  being  compared 

in  this  analysia.  This  is  of  considerable  help  to  managers  and  auditors’4- 

^Copies  of  the  computer  code  uaea  in  this  study  may  be  secured  from  the 
Center  for  Cybernetic  Studies  at  The  University  of  Texas  at 
Austin. 
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See,  e.g.,  Dennis  Thomas  £18]  for  a  discussion  of  ufte  use  of  DEA  m 
association  with  on-aite  management  audits  by  the  Texas  Public  Utility 
Commission  and  see  Churchill  et  al  £15]  for  an  interpretation  of  PSRQ 
(Professional  Standards  Review  Organizations)  as  auditors. 


(e.g.,  PSRO  review  panels)  as  well  as  analysts  when  evaluating  and 
investigating  the  behavior  of  particular  DMUs,  To  be  sure,  these  are  only 
estimates.  However,  DEA  appears  to  be  unique  among  evaluation  systems  not 
only  in  the  detai’,  it  supplies  on  each  DMU  but  also  in  the  fact  that  it 
evaluates  output  as  well  as  input  inefficiencies  in  each  of  them.  Note,  for 
example,  that  efficiency  could  be  achieved  by  DMU  24  in  Table  1  only  if  this 
facility  decreased  its  inputs  and  increased  its  outputs  in  the  indicated 
amounts.  That  is,  both  must  be  done  to  achieve  100X  efficiency. 

3.  MODEL  DEVELOPMENT  AND  DATA  COLLECTION 

The  collection  of  facets  used  to  obtain  those  evaluations  form  what  is 
called  an  "efficiency  evaluation  surface"  or,  more  briefly,  an  "efficiency 
surface".  To  continue  with  this  geometric  interpretation.-  we  say  regard  the 
output  and  input  data  for  each  DMU  as  corresponding  to  the  values  of  the 
coordinates  which  may  be  used  to  locate  (or  plot)  these  DMUs  in  a  space 
with  dimensioi  *  (or  axes)  consisting  of  the  three  outputs  and  aight  inputs 
included  in  this  study.  Mathematically  speaking,  the  DKA  procedures  we  have 
just  described  than  generate  an  "efficiency  surface”  which  envelops  these 
points.  Hence,  in  parr,  the  name  Data  Envelopment  Analysis. 

The  "lambda"  values  and  the  DMUs  to  which  they  pertain  in  Table  1,  for 
instance,  identify  a  point  on  this  efficiency  surface  from  which  DMU  24  is 
evaluated.  The  input  and  output  values  at  this  point  are  obtained  from  DMUs 
which  are  efficient.  Hence  the  alternative  thus  provided  can  be  said  to  be 
capablo  of  producing  what  was  produced  by  DMU  24  and,  indeed,  even  produce 
more  PERS  TRAIN  with  the  reduced  input  amounts  noted  under  "VALUE  IF  EFFI¬ 
CIENT"  in  Table  1.  All  inputs  may  thus  ha  reduced  without  reducing  any 
output.  Hence  no  substitutions  or  tradeoffs  to  be  madu  in  arriving  at 


a  .-judgment  that  DMU  24  is  inefficient  and  that  matters  will  be  improved  by 
proceeding  in  the  manner  portrayed  in  Table  1. 

All  technical  matters  such  as  generation  and  choices  cf  facets  for 
effecting  these  evaluations  are  handled  by  DEA  in  the  form  of  a  mathematical 
programming  model  and  related  computei  codes  that  are  already  available  for 
these  purposes.  This  leaves  analysts  and  managers  free  to  focus  on  the 
choices  of  inputs  and  outputs  to  be  utilized  and  the  choices  of  facilities 
such  as  hospitals  or  subdivisions  thereof  which  are  to  be  regarded  as  DMUs. 
See,  e.g.,  D,  Sherman  £173  for  a  discussion  or  his  reasons  for  choosing 
surgical  units  within  Massachusetts  hospitals  as  DMUs  rather  than  the 
hospitals  themselves  for  his  DEA  evaluations. 

In  the  present  case  only  a  prototype  study  was  undertaken  to  evaluate 
DEA  as  a  possible  addition  to  the  Army's  methods  for  evaluating  its  hospi¬ 
tals  and  other  health  services  facilities.  To  facilitate  interpretation  and 
evaluation  of  results  the  number  of  outputs  and  inputs  (along  with  the 
number  of  DMUs)  was  kept  to  a  relatively  small  number  so  that  computer 
printouts  and  other  results  could  be  more  easily  analyzed.  This  also  proved 
to  be  advantageous  for  checking  data  quality  ar.d  for  studying  alternative 
ways  m  which  the  data  might  be  utilized  as  well  as  alternative  types  of 
data  that  might  be  collected. 

An  example  of  what  might  be  done  with  further  extensions  is  provided  by 
reference  to  the  quality  of  health  care  services  delivered.  At  present  the 
Army  collects  data  on  both  patient  and  physician  satisfaction,  as  well  as 
other  attitudinal  data  that  bear  on  the  quality  of  services  rendered  and 
received.  Although  data  like  these  can  be  treated  as  outputs  or  as  inputs 
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(or  both)^  it  would  aeon  better  to  recognize  the  conplex  dimensions  of 
"quality"  and  to  add  additional  measures  that  can  help  to  capture  these 
additional  aspects  of  the  services  rendered.  For  instance,  data  on  facil¬ 
ities  obtained  from  PSRO  reviews,  might  be  used,  and  even  audit  follow-ups 
subsequent  to  discharge  might  be  used  to  generate  more  objective  data  which 
could  then  serve  to  complement  the  present  subjective  evaluations  of  quality 
for  use  in  a  more  extended  DEA  analysis. 

The  point  is  that  DEA  can  accomodate  the  interdependencies  associated 
with  overlapping  variables  even  though  such  interdependenc i es  are  often 
troublesome  to  handle  in  standard  forma  of  statistical  regressions.  DEA.  can 
be  (and  is)  reduced  to  an  ordinary  linear  programming  model  for  purposes  of 
computation.  Hence,  the  great  computational  power  and  efficiency  associated 
with  developments  in  linear  programming  is  available  and  can  be  brought  to 
bear  for  handling  large  numbers  of  inputs,  outputs  and  DMUs.  Finally, 
sensitivity  analysis  and  the  rich  interpretive  power  available  from  already 
completed  linear  programming  research  is  available  for  checking  data  quality 
in  terms  of  the  sensitivity  of  solutions  to  possible  errors  in  the  data. 
Hence  problems  of  data  collection  and  refinement  can  be  joined  to  the  r<s- 
search  for  model  development  in  a  twin  strategy  that  can  be  used  to  improve 
and  economize  both  data  collection  and  model  development  not  only  in  the 

2 

research  but  also  in  the  actual  use  phases  of  DEA  . 

^Data  which  appear  as  both  an  output  and  an  input  ars  sometimes  re¬ 
ferred  to  es  intermediate  good*.  See  Chapter  IX,  pp.  295  ff.  in  [7]  for 
more  precise  definitions  and  characterizations  of  "intermediate'*  goods  or 
processes . 

2 

See  C&]  for  a  detailed  descusaion  of  such  twin  strategies  for  data 
collection  and  model  development. 


12 


4.  WINDOW  ANALYSIS 


Still  other  Methods  of  testing  and  validation  are  available  which  have 
been  specifically  devised  for  use  in  DEA.  These  Methods  can  also  be  used  to 
provide  results  and  interpretations  which  are  of  interest  in  their  own 
right. 

Table  2  provides  an  example  of  such  a  DEA  development  which  is  referred 
to  as  a  "Window  Analysis".  The  procedures  for  obtaining  such  an  analysis 
aay  be  described  as  follows.  Recall,  first,  that  we  have  eight  quarterly 
reports  for  esch  of  the  24  DMUs  comprehended  in  this  study.  In  this  case  it 
was  decided  to  arrange  a  four-quarter  window  analysis  with  the  results 
displayed  in  Table  2. 

Focusing  on  DMU  1  in  Table  1  we  turn  to  the  first  row  and  observe  that 
there  are  four  values  ranging  frca  0.85  to  0.35.  These  are  utA  efficiency 
ratings  of  the  kind  shown  for  DMU  24  at  the  top  of  Table  1.  In  this  case, 
however.  DHU  1  is  treated  as  if  it  were  a  different  DMU  in  each  of  the  first 
four  quarters,  and  the  same  treatment  is  accorded  to  every  one  of  the  DMUs 
In  this  study.  Thu*  instead  of  24  actual  DHUa  we  have  4  x  24  =  96  DMUs  for 
which  •afficlency  ratings  are  to  be  secured.  The  efficiency  ratings,  one  for 
each  quarter,  are  exhibited  for  each  DMU  in  the  first  row  so  that  for  DMU  1 
we  have  ratings  of  0.85.  0,95,  0.85  and  0.94  in  quarters  1,  2,  3,  and  4, 
respectively,  as  shown  in  Table  1,  ' 

Now  auppose  the  first  quarter  data  are  eliminated  and  replaced  by  the 
fifth  quarter  for  each  DMU,  A  new  sat  of  efficiency  ratings  with  this  new 
set  of  DHUa  say  then  be  obtained  in  the  sane  manner  as  before.  This  pro¬ 
duces  the  efficiency  score*  ahown  in  the  second  row  (alongside  each  DMU)  in 


Table  2. 


TABLE  2 

EFFICIENCY  SCORES 


DNU  No. 


Q!  4-  ^3  84  Q3 

66  97  Q« 

1 

.gS  .93  .*3  • 94 

.98  .06  .93  .81 

*06  .96  .80 

.77 

.97  .77 

•73  .79 

.79 

•71  *80  #88 

2 

.80  .77  .72  .72 

•77  .72  .73  .84 

.69  .69  .7? 

.79 

•78  .72 

.78  .81 

•  72 

•77  »fil  *71 

3 

.93  .90  .91  .83 

•89  .91  .64  .91 

.88  .81  .87 

.90 

.76  r 86 

.93  .94 

•  83 

-94  1.00  1.00 

4 

.98  .93  .93  1.00 

•93  .93  1.00  .86 

•90  1.00  .81 

1.00 

1.00  .80 

1.00  .92 

•  81 

1.00  .91  .82 

5 

1.00  1.00  1.00  1.00 

l.oo  i.eo  i.oo  i.oo 

1.00  1.00  1.00 

.93 

.98  .95 

.83  1.00 

.41 

cf2  1 c35  l.SS 

6 

.19  .83  .84  .86 

•82  .83  .86  .93 

• 

•83  .83  .94 

.89 

•82  .as 

•84  1.00 

•  83 

•83  1.00  1.00 

7 

1,00  1.00  1.00  1.00 

1.00  1.00  1.00  .96 

1.00  1.00  .96 

l.OC 

1*00  .94 

.98  .98 

•  96 

1.00  1.88  1.00 

8 

•  99  #97  *98  1.00 

•  V7  871  j*VU  871 

•94  1.09  .90  1 

.00 

1.00  .90  1 

•00  ,9t 

•  94  1 

•00  ,98  .92 

9 

1.00  1.00  1*00  1.00 

.94  .97  1.00  l.OD 

.97  1.60  1.00  ] 

.00 

1.00  1.00 

.»*  1.00 

1.00  1 

.00  1.00  1.00 

10 

1.00  1*06  1.00  I.oo 

l.tl  1.00  1.00  1.08 

1.00  1.88  .99  1 

.00 

i.to  •»  i 

••«  l.OD 

11 

.92  .98  1.00  1.00 

.98  1.00  1.00  1.00 

l.OD  1. OB  1.00  1 

.OP 

1.00  1.00 

.97  1.00 

1.00 

•  98  1.00  .93 

1.00 

1.00 

l  .no 

1.00 

1.00 

1.00 

•  97 

1.00 

1.00 

•  91 

.94 

1.00 

•  90 

.93 

.87 

.92 

SUMMARY  MEASURES 

MEAN  VAR  COLUMN  TOTAL 

NANCE  RANSC 
•IS  *01  *04  *25 


.73  .0*  .12  .13 


-9*  «88  .97  .22 


•  93  .it  .66  .20 


•»?  •#•  -12  .10 


•88  .06  .12  .10 


•  99  .IIC  »«3  .06, 


*88  .04 


•99  .00  .C6 


•98  .88  .21 


•99  .98  .03 


.97  .00  .10 


DNU  No. 

13 

14 

15 

16 

17 

1  r> 

J.O 

19 

20 

21 

22 

23 

24 


EFFICIENCY  SCORES 


<u 

41 

43 

44 

4B 

46 

47 

Q8 

MEAN 

VAN 

COLUMN 

total 

.98 

.91 

1.00 

.99 

NANCE 

nance 

•  91 

1.00 

.98 

.93 

.96 

•  80 

.07 

•  11 

i.Ga 

.97 

.94 

*93 

.43 

.93 

.92 

.89 

.95 

•  96 

.96 

1.00 

DO 

1.00 

1.00 

i 

.00 

1.00 

1.00 

l 

.00 

1.00 

.99 

■  00 

•  08 

.08 

1.00 

i 

.00 

.97 

1.00 

i 

.PO 

•  9? 

•  99  I 

.00 

.9* 

.97  1 

.00 

1.00 

1*00  1.00 

1 


•  95  1.00 
1.00 


•93  l.oo 

.*6 


*v*  i, 


>96  l.Qo 


•  os 


.09 

.09 


« 

.73 


.07 

>07 


•93  1.00  i 
1.00  1, 


1.00  l.oo  1.00  1.00 
1.00  1.00  1.00 
l.oo  l.oo 
1.00 


5  .96  1.00  .90 

•97  1.00  .90  1.00 

1.00  .#9  |.C(}  1#00 

•89  ,98  1.00  1,00 

.97 

.00 

•  02 

•  11 

1  .96  .98 

•96  .98  1.00 

•93  ,8g  ,89  l.OP 

•90  .89  l.oo  l.oo 

.88  l.GO  ,98  1.00 

.96 

•  00 

•  1? 

.12 

1.00  .97 

•99  .93  .90 

1.00  .93  .89  .95 

•97  .84  .f  4  .95 

-A3  .91  ,91  l.oo 

.94 

•  on 

.07 

-17 

1.00  .93 

1.00  .99  ,9g 

1.00  1.00  .97  l.oo 

•98  .95  l.OP  l.oo 

•93  .99  1.00  1.00 

•  99 

.00 

.05 

.07 

•  89  .73 

•A"  -73  .07 

*AA  -7J  .86  .84 

•76  .87  .02  i.op 

•* 7  *40  1.00  1.00 

.86 

.01 

.04 

«  ?  7 

1  .  JO  l.oo 

1.00  1.00  1.00 
•00  1.00  l.oo  l.oo 

1.00  l.oo  l.oo  .97 

1.00  l.oo  ,98  J.oo 

iv  00 

.00 

.00 

.05 

•80  l.OQ 

•00  1.00  .93 

•00  1.0(5  .95  ,,D0 

1.00  .91  |,O0  l.oo 

.93  1,00  1.00  1.00 

.98 

.00 

.03 

\ 

.09 

.80  .77 

•80  .77  .77 

"8n  *76  .75  .77 

-76  .7J  ,70  .74 

,M  »T5  .74  ,78 

•  77 

.00 

*04 

.14 

.00  .96 

CO  .96  l.o# 

00  -96  .97  1.00 

*•00  .93  1.C0  l.oo 

•’1  .99  1.00  1.00 

•  98 

.00 

.09 

.09 

.89 

.87  1.00' 

• OJ  .99  1.00 
•O*5  -91  1.00  1,00 


•97  .00  .09 
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Returning  to  Table  1  we  can  now  relate  the  facet  ae*bera  listed  at  the 
top  of  this  table  in  the  following  way:  DMU  71  the  first  *e»ber  listed  in 
the  row  labelled  FACET  is  actually  DMU  23  represented  by  its  third  quarter 
data;  DMU  87  represents  DMU  15  in  teras  of  its  fourth  quarter  behavior;  and 
so  on.  To  further  reenforce  what  we  are  saying  we  sight  particularly  note 
that  DMU  95  is  actually  DMU  24  itself  in  the  fourth  quarter  so  that  DMU  24 
is  being  evaluated  in  this  report  for  its  second  quarter  behavior  by  re¬ 
ference  to  efficient  DMUa  that  include  DMU  24  in  the  third  quarter. 

We  thus  obtain  an  expanded  collection  of  DMUa  which  can  be  used  to 
obtain  further  insight,  into  their  behavior  and  the  DEA  efficiency  ratings 
that  are  accorded  to  the*.  Proceeding  in  the  Manner  we  have  just  indicated 
provides  an  overlap  or  “window**  which  can  be  used  to  study  the  stability  of 
these  DEA  ratings  fro*  the  different  sets  of  DMUa  that  for*  the  reference 
sets  for  the  rating*  in  the  different  rows.  Notice,  for  instance,  that  the 
values  for  DMU  1  sre  relatively  stable  in  each  of  the  colu*ns  (windows)  that 
eight  be  for*ed  to  exaeine  its  behavior  even  though  different  collections  of 
DMUa  ara  available  for  forcing  the  efficient  reference  set.  We  can  there¬ 
fore  say  that  its  efficiency  rating*  are  stable  in  each  colusn  even  though 
different  reference  seta  are  involved. 

Of  course  the  row  behavior  ia  also  of  intereet.  Notice,  for  instance, 
that  the  efficiency  values  for  ail  DMUs  in  colusn  Q5  of  Table  2  tend  to  be 
low  relative  to  the  values  in  tho  other  colu*ns.  This  reflects  an  "after- 
Chriataas'*  leg  in  productivity  and  low  workload. 

Data  inadequacies  and  possible  data  errors  can  also  be  detected  in  this 
manner.  For  instance  largo  guaps  in  efficiency  values  within  a  coluan  nay 
result  fro*  events  such  as  a  new  policy  or  turnover  and  replacesent  of 
iaportent  aenbers  of  the  staff  in  a  particular  DMU.  Such  behavior  can  also 


result  fro*  mlsraported  or  erroneous  data  and  these  possibilities  also  need 
to  be  considered.  See  13]  and  £51  for  further  discussion  in  DEA  applica¬ 
tions  to  Maintenance  activities  of  the  U.S.  Air  Force.  Sherman  £17]  also 
reports  similar  findings  of  erroneous  data  and  reports  in  his  DEA  study  of 
surgical  units  in  Massachussetts  hospitals. 

The  means,  variances,  and  ranges  on  the  right  in  Table  2  may  be  used  as 
guides  to  where  underlying  DMU  behavior  might  be  studied  further  and  perhaps 
arrayed  in  more  detail  in  a  fora  such  as  was  exhibited  in  Table  1  for  this 
purpose.  These  values  can  also  prove  useful  in  their  own  right.  For 
instance  the  mean  DEA  efficiency  of  0.75  shown  on  the  right  of  Table  2  for 
DMU  2  is  lower  than  the  ratings  for  any  of  the  other  DMUs.  Looking  into 
this  result  in  further  detail,  however,  ahowed  that  the  activities  at  this 
facility  involved  extensive  residency  ana  intern  training.  The  PERS  TRAIN 
reported  in  connection  with  Table  1  refers  to  military  training  and  hence 
does  not  adequately  reflect  these  other  important  services  provided  by  DMU 
2.  Thus  this  rendering  of  a  low  efficiency  score  for  DMU  2  cannot  be 
regarded  as  wholly  valid  without  the  addition  of  these  kinds  of  outputs  and 
so  a  future  rerun  of  this  study  should  expand  the  number  of  outputs  to 
include  residencies  and  Intern  training  if  facilities  like  DMU  2  are  to  be 
covered . 
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5. EXTENSIONS  AND  CONCLUSIONS 


The  preceding  analyses  and  discussions  are  intended  to  provide  some 
Insight  into  the  nature  of  DEA  as  well  as  the  study  being  undertaken  by  the 
Army  Health  Services  Command.  In  both  cases  only  minimal  summaries  were 
given.  For  instance  on  the  DEA  side  the  discussions  accepted  the  present 
level  of  activities  at  each  DMU  without  considering  the  possibility  of 
reallocating  resources  between  the  different  facilities.  The  latter 
questions  may  involve  a  need  to  consider  possible  “economies  of  scale”  for 
each  DMU  along  with  possible  further  tradeoffs  in  the  input  and  output  mixes 
that  each  DMU  might  use. 

The  lambda  valuea  shown  at  the  top  of  Table  1  can  also  be  used  for  this 
purpose.  In  particular,  as  shown  in  Banker  til, DEA  identifies  a  DMU  as 
being  in  a  region  of  increasing ;  constant  or  decreasing  returns  to  scale 
according  to  whether  the  sum  of  the  lambda  values  exceeds,  is  equal  to,  or 
less  than  unity.  Thus,  here  again  DEA  supplies  information  that  is  not 
readily  available  from  the  accounting  and  ratio  controls  customarily  used  by 

regulatory  and  other  agencies,1'  To  illustrate  we  return  to  Table  1  and  find 
from  the  lambda  values  listed  at  the  top 

0.373  ♦  0.224  ♦  0.254  ♦  0.313  «•  0.132  ♦  0.148  *  1.449. 

Since  the  sum  of  these  lambda  values  exceeds  unity,  we  find  that  DEA  iden¬ 
tifies  DMU  24  to  be  operating  in  a  region  of  increasing  returns  to  scale. 


*See  Sherman  1173  for  a  detailed  discussion.  An  analysis  of  some  of 
ths  difficulties  in  the  use  of  stetistical  regressions  may  be  found  in 
Feldstein  (163  although  recent  developments  may  help  to  alleviate  some  of 
these  diff iculties.  See  Chernes,  Cooper  and  Sueyoshi  (143. 


To  make  this  sore  precise  we  nay  note  that  this  increasing  returns  to 
scale  is  associated  with  the  conparison  point  for  DMU  24  on  the  efficiency 
surface.  Recall  that  this  is  the  point  generated  fron  the  “lambda"  values 
applied  to  the  facet  Members  listed  at  the  top  of  Table  1.  The  adjustments 
to  obtain  the  "VALUES  IF  EFFICIENT"  in  Table  1  position  DMU  24  on  this 
point.  The  fact  that  the  “lambda"  values  sum  to  a  value  greater  than  unity 
means  that  if  these  efficient  input  values  are  increased  in  a  given 
proportion  the  resulting  movement  on  this  surface  will  produce  a  more  than 
proportionate  increase  in  the  efficient  output  values  on  thi3  part  of  the 
efficiency  surface.  (Conversely,  if  the  inputs  were  all  decreased  in  a 
specified  proportion  then  the  outputs  would  decrease  sore  than 
proportionately. ) 

Results  like  these  can  be  of  considerable  interest  to  the  Army  in 

•  • 

budgeting  and  planning  its  health  care  activities.  The  ability  of  DEA  to 
identify  such  additional  possibilities  for  each  DJUJ  after  efficiency  haa 
been  achieved,  makes  it  an  important  alternative  to  the  statistical  regres¬ 
sion  techniques  that  are  often  used  to  study  the  presence  of  returns  to 
scale  possibilities.  Banker  et  al  in  (23 ,  for  example,  redid  a  previous 
regression  study  of  North  Carolina  hoapitals  for  returns  to  scale  possi¬ 
bilities.  Applying  DEA  to  exactly  the  same  data.  Banker  and  his  associates 
were  able  to  identify  many  returns  to  scale  situations  which  the  previous 
regression  study  had  "average  out"  to  reach  a  conclusion  that  no  such  pos¬ 
sibilities  ware  present. 

Of  course  these  types  of  DEA  characterization*  and  findings  need  fur¬ 
ther  study  and  validation  before  they  are  put  into  use.  We  have  described 
only  some  of  the  ways  in  which  it  is  being  examined  in  the  present  study. 


Other  uaea  and  testa  are  a *30  possible,  of  course,  and  these  include  de¬ 
tailed  comparisons  with  other  approaches  that  are  familiar  from  past 
experience. 

The  following  figure  may  be  of  interest  and  some  of  the  insights  it 
provides  aay  help  ua  to  close  this  paper  on  a  note  that  will  help  to  indi¬ 
cate  some  of  the  further  possibilities.  In  this  case  the  inpatient  cost3 
per  Relative  Weighted  Product  (RWP)  as  defined  for  output  2  in  section  2, 
above,  had  already  been  extensively  studied  (but  just  for  inpatient  costs) 
by  aeana  of  standard  types  ox  regesalon-correlation  analyses.  This  there¬ 
fore  provided  a  convenient  coaparison  test  for  DEA,  parts  of  which  are 
incorporated  in  the  following  figure. 

The  points  in  this  figure  represent  DEA  estimates  of  inpatient 

costa/RWP  of  selected  DHUs  fro*  windows  1  and  5.  These  costs  are  obtained 

* 

on  the  basis  of  efficient  operations  by  first  making  tho  adjustments  to 
"VALUE  IF  EFFICIENT"  in  the  manner  described  for  Table  1.  The  plus  signs 
represent  regression  estisates  for  inpatient  costs/RWP  at  these  same  DMUs  as 
obtained  from  the  previously  conducted  regression  studies. 

For  the  eight  facilities  selected  from  the  regression  studies  and 
represented  as  plus  signs  in  this  figure  it  may  be  observed  that  six  of  them 

agree  with  the  DEA  results,1  The  high  regression  value  within  the  range  of 
15  RWP  had  some  specialities  not  normally  associated  with  a  facility  of  this 
size  --  i.e.,  extensive  oral  surgery,  gastroenterology  and  a  large 
orthopedic  service  --  while  the  high  regression  value  within  the  range  of  <*0 


1 A  further  detailed  discussion  and  comparison  between  regressions  and 
DEA  via  a  simulation  study  may  be  found  in  [4J  which  used  the  kinds  of 
regresaions  discussed  in  C163  for  its  models. 
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RWP  ia  associated  with  DMU  2  which  has  the  exceptional  status  that  was  noted 
earlier. 

The  dashed  line  shown  in  the  figure  represents  an  approximation  to  the 
HCFA  reimbursement  level  using  the  HCFA  unadjusted  Federal  rate.  Inter' 
preted  as  an  efficiency  measure  (above  which  compensation  will  not  be  paid), 
it,  together  with  the  regression,  identifies  DMU  2  as  inefficient  by  refer¬ 
ence  to  the  regression  estimate  indicated  by  the  plus  sign  near  40  RWP. 
This  is  perhaps  to  be  expected  since,  as  indicated  by  the  horizontal  line, 
this  HCFA  level  ia  applicable  without  respect  to  volume  or  mix  consider¬ 
ations. 

Returning  to  the  issue  of  increasing  returns  to  acale,  it  is  of  inter¬ 
est  to  see  that  this  is  generally  born  out  by  the  fact  that  the  per  unit 
costs  tend  to  fail  with  increasing  RWP  as  indicated  by  the  DEA  efficiency 
values  represented  by  the  points  in  the  diagram.  To  be  sure,  this  13  only 
indicative  of  such  increasing  returns  to  scale  3ince  the  increases  in  RWP 
for  the  DMUs  represented  in  this  figure  reflect  variations  in  both  input  and 
output  mixes  as  wall  as  volume  changes  in  the  inputs.  This  behavior  is 
consistent  with  the  DEA  returns  to  scale  characterizations,  however,-  and  it 
should  be  noted  that  such  behavioral  characterizations,  which  represent 
additional  opportunities  for  increasad  efficiency,  are  not  identified  in 
either  the  regression  values  or  the  HCFA  reimbursement  standard. 

Still  other  possibilities  can  ba  (and  have  been)  undertaken  in  the 
present  study  of  DEA  and  more  are  still  to  ba  undertaken.  As  noted  in  a 
recent  report  to  the  Army's  Health  Care  Studies  and  Clinical  Investigation 
Activities:  DEA  has  generated  reasonable  results  for  the  efficient  surfaces 
used  to  evaluate  multiple  input  and  multiple  output  systems  with  very  com- 
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plex  interrelationships.  Although  further  analysis  is  required,  the  pre¬ 
liminary  work  to  date  shows  that  DEA  holds  great  promise  as  a  method  to 
quickly  identify  faulty  data  and  to  summarize  an  efficiency  surface  with 
advantages  for  identifying  and  estimating  possible  ineff iciencies.  The 
limitations  of  regression  and  other  types  of  analyses  for  dealing  with 
auitipie  outputs  and  inputs  are  well  known  and  DEA  appears  to  overcome  these 
deficiencies.  Uses  of  DEA  and  regression  and  other  techniques  in  complemen¬ 
tary  fashion  are,  of  course,  also  possible.  In  any  case  DEA  appears  to  be 
an  extremely  valuable  technique  and  should  be  considered  for  use  by  tne 
medical  manager  and  for  further  research  as  part  of  medical  information 


science. 
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